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(54) Image signal encoding device. 

(57) An image signal encoding device which in- 
cludes a code transforming unit. The code 
transforming unit includes a separation unit 
which separates mode information from first 
code data, a locally decoding unit which locally 
decodes the first code data by means of a 
decoding operation con-esponding to the mode 
information and outputs locally decoded pixel 
block, and a second encoding unit which en- 
codes the locally decoded pixel block by means 
of an encoding operation corresponding to the 
mode information and outputs second code 
data. According to the image signal encoding 
device, the first code data is re-encoded into the 
second code data having a higher compress- 
ibility so as to perform efficient data transmis- 
sion and storage. 
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BACKGROUND OF THE INVENTION 

1. Field of the Invention 

5 The present invention relates to a device for encoding an image signal. 

2. Description of the Related Art 

In a device for receiving a manuscript as a digital image signal and performing an editing processingjand 
ro the like on the manuscript digital image signal, there is necessary a page memory which is used to store the 
data of the image temporarily. 

In recent years, the capacity of the page memory tends to increase as the resolution of the image to be 
processed and the number of the colors of the image increase. This leads to the two following problems. 
That is. one of the problems is that the price of the page memory is increased, while the other problem 
(5 is that there are increased a time necessary to read out an image from the page memory as well as a time 
necessary to write the image into the page memory. 

To solve these problems, there is proposed an image encoding technique which aims at reducing a page 
memory, as disclosed in Japanese Patent Unexamined Publication No. Hei. 5-56282. 

The method disclosed in the above-mentioned Japanese Patent Unexamined Publication No. Hei. 5-56282 
20 aims mainly at satisfying the three following requirements. 

That is, one of the requirements is that an image can be encoded at a uniform compressibility indepen- 
dently of the image. 

A second requirement is that an image divided into a given number of units can be encoded in a given 
amount of codes. 

25 A third requirement is that an encoding/decoding processing can be executed at a high and constant speed. 

Description will be given below of an image encoding device which is disclosed in the above-mentioned 
Japanese Patent Unexamined Publication No. Hei. 5-56282. 

Fig. 12 is a block diagram showing the structure of the image encoding device. In Fig. 12, reference nu- 
meral 1 designates an input image which is input by a raster scan shown in Fig. 1 3, 3 an input block consisting 
30 of mxn pixels converted or transformed by a blocking section 2 as shown in Fig. 1 3, 5 average value information 
which is calculated by an average value separator 4, 6 an average value separation block obtained by sub- 
tracting an average value from the input block 3 by the average value separator 4, 8 an approximate parameter 
or mode information obtained by a mode discriminator 7, 10 an approximate block obtained when an adaptive 
approximation encoder 9 performs a resolution approximation and a gradation approximation on the average 
35 value separation block 6, and 12 code data which is multiplexed by a multiplexer 11. The blocking section 2 is 
used to convert the scan order of the input image 1 from a raster scan into a blocking format as shown in Fig. 
1 3, the average value separator 4 is used to calculate the average value of the input block 3 and subtract the 
average value from the input block 3, the mode discriminator 7 is used to analyze the statistic and spatial char- 
acteristic amounts of the average value separation block 6 and to obtain an approximate parameter in approx- 
40 imation encoding in accordance with the results of the analysis, the adaptive approximation encoder 9 per- 
forms a resolution approximation and a gradation approximation, and the multiplexer 11 is used to multiplex 
the average value information 5 and mode information 8 as well as the approximation block 10. 

Now, in Fig. 14, there is shown a block diagram of the structure of the mode discriminator 7 shown in Fig. 
12. In Fig. 14, reference numeral 16 designates a vector index showing a representative vector which is selected 
45 by a pattern matching device 15. 18 a parameter candidate value, that is, waveform infomiation relating to a 
resolution approximation to be output from a waveform mapping table 17, 23 a variance value which is output 
from a variance calculator 22, 25 histogram information which is output by a histogram counter 24, 27 a para- 
meter candidate value, that is, gain information relating to the gradation approximation, and 14 a waveform 
analyzer which is used to analyze the waveform information of the statistic and spatial characteristic amounts 
50 as shown in Fig. 15. The pattern matching device 15, is used to perform a pattern matching between the set 
of the representative vector having representative waveform information and the average value separation 
block 6 consisting of mxn pixels to thereby select a representative vector having the nearest waveform infor- 
mation, the waveform mapping table 17 obtains a parameter candidate value on the resolution approximation 
from the vector index 16, a gain analyzer 20 is used to analyze the gain information in the block consisting of 
55 mxn pixels shown in Fig. 13 out of the statistic and spatial characteristic amounts, the variance calculator 22 
is used to calculate the variance value of the value of mxn pixels of the average value separation block 6, the 
histogram counter 24 counts the frequency distribution of the value of mxn pixels of the average value sepa- 
ration block 6, a gain mapping table 26 obtains a parameter candidate value on the gradation approximation 
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from the variance value 23 and histogram information 25. and a mode judging device 21 obtains the mode 
information 8. that is. an approximate parameter from the waveform information 18 and gain information 27. 

Referring now to Fig. 16. there is shown a block diagram of the structure of the adaptive approximation 
encoder 9 shown in Fig. 12. In Fig. 16. reference numeral 33 designates a sub-sample block which is sub-sam- 
5 pled by an adaptive sub-sampler 32. and 30 a sub-sample pattern generator which, in accordance with the 
parameter on the resolution approximation of the mode information 8. obtains a sub-sample pattern 31 for con- 
trolling a sub-sampling operation to be performed of the resolution approximation. The adaptive sub-sampler 
32 sub-samples the mxn pixels of the average value separation block 6 in accordance with the sub-sample 
pattern 31 to be output from the sub-sample pattern generator 30, and an adaptive quantizer 34 carries out a 
10 gradation approximation operation, that is, quantizes the gradation approximation in accordance with the para- 
meter on the gradation approximation of the mode information 8. 

Next, description will be given below of the operation of the above-mentioned image encoding device with 
reference to Figs. 12. 14, and 16. As shown in Fig. 13, the input image 1 which is input by the raster scan is 
scan converted into the input block 3 consisting of mxn pixels by the blocking section 2. After then, all encoding 
15 processings are performed independently of one another in blocks with the input block 3 as a unit. 

The average value separator 4 shown in Fig. 12 calculates the average value of the value Si| (i=1. 2, - 
-, m. j=1, 2, — . n) of the mxn pixels forming the input block 3. that is, the average value information 5 and, 
after then, subtracts the average value n from the respective pixel values of the mxn pixels forming the input 
block 3. and calculates the average value separation block 6 from the values X,, (i=1. 2. — . m, j=1. 2. — . n) 
of the mxn pixels that have received the average value separation processings. Here, Si,, (.i and Xn are ex- 
pressed by the following mathematical equation: 



20 



30 The mode discriminator 7 shown in Fig. 12 analyzes the statistic and spatial characteristic amounts of the 

average value separation block 6 and outputs the mode information 8, that is, the approximate parameter in 
accordance with the analysis results. As shown in Fig. 14, the waveform analyzer 14. which is a component 
of the mode discriminator 7. analyzes the waveform information to obtain the candidate value of the resolution 
approximate parameter. Likewise, the gain analyzer 20 analyzes the gain information to thereby obtain the 
35 candidate value of the gradation approximation parameter. Similarly, the mode judging device 21 obtains the 
approximate parameter, that is. the mode information 8 from the resolution approximate parameter candidate 
value and gradation approximate parameter candidate value. 

The waveform analyzer 14 shown in Fig. 14 analyzes the waveform information that represents the two- 
dimensional direction and complexity of the gradation variations of the average value separation block 6 and. 
40 from the analysis results thereof, obtains a resolution approximate parameter candidate value for approxima- 
tion of the resolution of the resolution of the average value separation block 6. 

The pattern matching device 1 5 which is shown in Fig. 14. for example, as shown in Fig. 1 5, pattern match- 
es a representative vector set including previously'prepared representative wavefonn information with an ana- 
lysis target block (which will hereinafter be referred to as an analysis block), that is. the average value sepa- 
ls ration block 6 to thereby carry out a waveform information analysis. Thus, the direction and complexity of the 
gradation variations of the analysis block can be obtained by means of the waveform information analysis, 
while the vector index 16 can be obtained from the index of the representative vector. When an analysis block 
of mxn pixels is expressed as x={Xi I i=1 . 2. — . mxn} and a representative vector set consisting of k pes. of rep- 
resentative vectors is expressed as Y={yi I i=1 . 2, — . k}, t hen a pattern matching can be defined by the following 
50 equation. 

For every i, d(x,yp)=min{d(x.yi)} (i=1 , 2, — , k). where d(x.yi) is a distortion measure and can be defined by 
means of a square distortion or the like. & is the index of a representative vector, that is, the vector index 16. 
and this shows that a representative vector Xp represented by £ has been selected as the representative vector 
that includes the nearest waveform information to the analysis block. 
55 The waveform mapping table 17 shown in Fig. 14 obtains a resolution approximate parameter candidate 

value, that is, obtains the waveform information 1 8 from the vector index 1 6. The resolution approximate para- 
meter candidate value consists of the direction v(p) of the analysis block representing the two-dimensional 
direction of the gradation variations and a sub-sample rate r(p) representing the complexity of the gradation 
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variations. 

The gain analyzer 20 shown in Fig. 14 analyzes the gain information that represents the frequency dis- 
tribution of the amplitudes and pixel values of the average value separation block 6 and. from the analysis re- 
sults thereof, obtains a gradation approximate parameter candidate value for approximation of the gradation 
of the average value separation block 6. The gain information analysis can be performed by counting the va- 
riance value a2 of the values of the mxn pixels constituting the average value separation.block 6 and the his- 
togram thereof. 

The variance calculator 22 shown in Fig. 14 calculates the variance value ct^ of the values of t be mxn pixels 
constituting the average value separation block 6, that is. the variance value 23. The variance value ofjthe 
mxn pixels with the average value separated therefrom can be defined by the following equation. 
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Or. description will be given hereinafter by use of the variance value ct. 



The histogram counter 24 shown in Fig. 14, as shown in Fig. 17. performs a threshold value processing 
on the average value separation block 6 by means of the variance value a to thereby count the frequency. 
That is. the threshold values are set to tola and the frequency is counted in three ranges consisting of a range 
of less than -a/a. a range of -a/a to a/a. and a range of more than a/a. Here, a is a real number which is 1 or 
more and, in the present embodiment, for example, a=3. The frequency values counted in the three ranges 
are expressed as H.,, Ho, and H,. respectively. As shown in Fig. 17. from H.,. Hq. and H„ it is judged whether 
the histogram is a unimodal distribution or a bimodal distribution and the result of this judgement is obtained 
as the histogram information 25. For example, when H.,SHo and HoSH,. then the histogram is judged as a 
unimodal distribution and. in other cases, the histogram is judged as a bimodal distribution. 
35 The gain mapping table 26 shown in Fig. 1 4 obtains a gradation approximate parameter candidate value, 

that is, the gain information 27 from the variance value 23 and histogram information 25. The gradation ap- 
proximate parameter candidate value consists of a gradation approximation, that is. the characteristic c of 
quantization and the number of levels 1. 

The mode judging device 21 shown in Fig. 14 obtains an approximate parameter, that is. the mode infor- 
^ mation 8 from the resolution approximate parameter candidate value and gradation approximate parameter 
candidate value. When performing a fixed length encoding operation to control an amount of codes to a given 
amount in each of mxn pixels, by operating the sub-sample rate r of the resolution approximate parameter can- 
didate value in connection with the code amount and the number of levels 1 of the gradation approximate para- 
meter candidate value, the data amount of the approximate block 10 is set to a given amount. Here, the data 
« amount of the approximate block 10 is proportional to p logz 1 and, therefore, the value of p lOQz 1 may be 
controlled to a given value. The thus controlled resolution approximate parameter and gradation approximate 
parameter are combined together and are then output as the mode information 8. Also, if the respective re- 
solution approximate parameter candidate value and gradation approximate parameter candidate value are 
used as the mode information 8 as they are. then the quality of the reproduced image can be maintained at 
50 a given level. 

The adaptive approximation encoder 9 shown in Fig. 1 2 carries out the approximation of the resolution as 
well as the approximation of the gradation in accordance with the mode information 8. 

The sub-sample pattern generator 30 shown in Fig. 16 obtains the pattern 31 for sub-sampling the average 
value separation block 6 in accordance with the gradation approximate parameter of the mode information 8. 
55 that is. the analysis block direction v which represents the two-dimensional direction of the gradation variations 
as well as the sub-sample rate r which represents the complexity of the gradation variations. The pattern 31 
is determined by combination of'the direction v with the rate r. For example, the pattern 31 includes a pattern 
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in which a block consisting of mxn pixels is thinned out to 1/2 only in the longitudinal direction thereof, a pattern 
in which the block is thinned out to 1/2 only in the transverse direction thereof, a pattern in which the block is- 
thinned out to 1/2 in both longitudinal and transverse directions thereof, and other patterns. 

The adaptive sub-sampler 32 shown in Fig. 16 sub-sannples the average value separation block 6 in ac- 

5 cordance with the sub-sampling pattern 31 . 

The adaptive quantizer 34 shown in Fig. 16 quantizes the sub-sample block 33 in accordance with the 
gradation approximate parameter of the mode information 8. that is. the characteristic c of the quantization 
and the number 1 of the quantization levels. As shown in Fig. 17. when the quantization characteristic c is a 
unimodal distribution, there is selected a quantizer which is ideally suited for the unimodal distribution...3nd. 

10 when the quantization characteristic c shows a bimodal distribution, there is selected a quantizer which is ideal- 
ly suited for the bimodal distribution. Here, as the quantizer, for example, there may be used a non-linear quan- 
tizer such as an optimum quantizer which performs a quantizing operation in view of the maximum value of 
distribution for each distribution. Quantizers should be prepared in accordance with the combinations of the 
quantization characteristic c with the quantization level number 1, respectively. 

15 The multiplexer 11 shown in Fig. 12 multiplexes the mode information 8, average value information 5 and 

approximate block 10 to thereby make up the code data 12 as shown in Fig. 18. 

On the other hand, as a highly efficient encoding method for communication and storage, there is known 
an encoding method on the basis of a Discrete Cosine Transformation (which will hereinafter be abbreviated 
to DCT) which is a kind of orthogonal transformation {Hideo Hashimoto: Introduction to Image Compression 

20 "Image Encoding Algorithm II -Transformation Encoding-", The Journal of the Institute of Television Engineers 
of Japan. Vol. 43. No. 10, 1989, pp. 1145-1152.) 

Description will be given below of the orthogonal transformation. For an image signal, there is employed 
a two-dimensional orthogonal transformation which uses a correlation between horizontal and vertical direc- 
tions. There is constructed a pixel block which includes M pixels and N pixels in the horizontal and vertical 

25 directions respectively, and one-dimensional orthogonal transformation is carried out in the horizontal and 
vertical directions independently of each other. These processings can be expressed by the following equation 

(1): 

Y = AnXAJ, (1) 

where X is a pixel block consisting of N rows and M columns, Y is a coefficient of transformation, and A^, Am 
30 are orthogonal transformation matrixes respectively of degree N and degree M. 

There are available various kinds of orthogonal transformation methods and. in view of encoding efficien- 
cy, the DCT method is generally used. For example, a two-dimensional DCT of degree N can be given by the 
following equation (2), while inverse transformation thereof can be given by the following equation (3). 
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where X{j.k) represents the respective elements of the pixel block 102 and j, k respectively express the posi- 
tions of the elements. Y(u.v) represent the respective elements of the transformation coefficients and u, v ex- 
press the positions of the elements. 
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In an image signal called a natural image such as a person, background and the like, it is known that mu- 
tually adjoining pixels tend to take near pixel values and have a high correlativity. Such a signal of high cor- 
relativity means, when viewed on a frequency axis, that a signal electric power is distributed collectively to a 
specific frequency component. By encoding only the coefficient of the component to which the signal power 

5 is distributed collectively, it is possible to reduce the amount of information as a whole. In the natural image, 
by carrying out the DCT method, most of the signal power is centered on a low frequency area. 

However, in the DCT method, there are known problems that the encoding efficiency is lowered if an edge 
is included in the image, and that specific image quality deterioration occurs in the neighborhood of the edge 
of a decoded image. This is because, in a pixel block including an edge, the signal power is not centeretd on 

10 a specific coefficient but is dispersed over to a wide range of coefficients. A method of reducing the image 
quality deterioration when such an image including an edge is encoded according to the DCT method is dis- 
closed in Japanese Patent Unexamined Publication No. Hei. 5-63989. 

In the method disclosed in Japanese Patent Unexamined Publication No. Hei. 5-63989, at first, the wa- 
veform of the pixel block is analyzed by use of pattern matching and variance calculation. Next, there is se- 

15 lected from the analysis results mask information which decides the significance/insignificance of the DCT 
coefficients. In the DCT coefficients, a coefficient at a position where the mask information shows insignifi- 
cance is made 0 forcibly. After the above coefficient select processing, entropy encoding is executed. 

The method disclosed in the above-mentioned Japanese Patent Unexamined Publication No. Hei. 5- 
56282. however, has two problems as follows: 

20 Firstly, in an image, there exist variations such as character areas, blank space areas and the like which 

are present locally and have a statistic character. However, in the method disclosed in the above Japanese 
Patent Unexamined Publication No. Hei. 5-56282, for the purpose of keeping a two-dimensional position re- 
lationship between the image and code data, the image divided into a given number of unit sub-portions is 
encoded in a constant amount of codes. For this reason, there exists redundancy in the encoded data. 

25 Secondly, because the method is developed mainly for internal use of the encoding device, it is not com- 

patible with other encoding methods which are used externally of the encoding device. 

Therefore, when the method disclosed in the above Japanese Patent Unexamined Publication No. Hei. 
5-56282 is used for communication and storage, it is expected that the code data encoded according to the 
method is once decoded and after then the decoded data is encoded again with high efficiency. 

30 However, due to the fact that the method disclosed in the above Japanese Patent Unexamined Publication 

No. Hei. 5-56282 uses an algorithm based on the block approximation encoding, an edge in the decoded image 
is approximated by a small number of representative gradations, with the result that the edge is emphasized. 
For this reason, it is not a wise way to encode again the decoded image directly according to the DCT method 
from the viewpoint of efficiency and image quality. 

35 

SUMMARY OF THE INVENTION 

The present invention aims at eliminating the problems found in the above-mentioned methods and de- 
vices. Accordingly, it is an object of the invention to provide an image signal encoding device which can trans- 
40 form the code data into the code data that can be decoded with an improved efficiency and by a decoding 
device provided externally of the present encoding device. 

Also, it is another object of the invention to provide an image signal encoding device which uses the results 
of analysis of the characteristics of pixel distributions by pixel blocks contained in the code data to thereby be 
able to reduce the deterioration of the image when the image is re-encoded. 
45 In attaining the above-mentioned objects, the invention provides an image signal encoding device includ- 

ing: dividing means for dividing an image signal into a plurality of rectangular pixel blocks; average value sep- 
arator means for calculating an average value of pixels in each of the pixel blocks and subtracting the average 
value from each of the values of the pixels of the pixel block; mode discriminator means for analyzing charac- 
teristics of resolution and gradation variations in an average value separation pixel block obtained by the aver- 
so age value separator means and outputting mode information which is defined in correspondence to the char- 
acteristics of the pixel distribution in the average value separation pixel block; first encoding means for encod- 
ing the average value separation pixel block by means of an encoding operation which is defined by the mode 
information; multiplexer means for multiplexing the encoded results output from the first encoding means, the 
average value output from the average value separation means, and the mode information and outputting first 
55 code data; separation means for separating the mode information from the first code data; locally decoding 
means for locally decoding the first code data by means of a decoding operation corresponding to the mode 
information and outputting a locally decoded pixel block; and second encoding means for encoding the locally 
decoded pixel block by means of an encoding operation corresponding to the mode information and outputting 
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second code data. 

BRIEF DESCRIPTION OF THE DRAWINGS 

5 Fig. 1 is a block diagram showing the structure of a first embodiment of an image signal encoding device 

according to the invention; 

Fig. 2 is a block diagram showing the structure of a code transforming unit employed in the first embodi- 
ment according to the invention; 

Fig. 3 is a block diagram showing the structure of a locally decoding unit employed in the first embodiment 
10 _ according to the invention; 

Fig. 4 is a block diagram showing the structure of a second encoding unit employed in the first embodiment 

according to the invention; 

Fig. 5 is a view showing scan transformation; 

Fig. 6 is a table showing examples of the distribution of pixel values in a block and the distribution of DOT 
15 coefficients; 

Fig. 7 is a view for explaining coefficient selection in the case of a two-dimensional orthogonal transfor- 
mation of degree 8: 

Fig. 8 is a view showing a transformation coefficient block and a coefficient scan sequence; 
Fig. 9 is a table showing groups by differences in direct current components; 
20 Fig. 10 is a table showing encoding symbols in alternating current components: 

Fig. 11 is a block diagram showing the structure of a second embodiment of the image signal encoding 
device according to the invention; 

Fig. 12 is a block diagram showing the structure of a proposed image signal encoding device; 
Fig. 13 is an explanatory view showing the scan transformation of an image: 
25 Fig. 14 is a block diagram showing the structure of a mode discriminator employed in the proposed image 

signal encoding device: 

Fig. 15 is a view for explaining the waveform information analysis; 

Fig. 16 is a block diagram showing the structure of an adaptive approximation encoder employed in the 
proposed image signal encoding device; 
30 Fig. 17 is a view for explaining the gain information analysis: and 

Fig. 18 is an explanatory view showing the code structure. 



DESCRIPTION OF THE PREFERRED EMBODIMENTS 



35 Fig. 1 is a block diagram showing the structure of a first embodiment of an image signal encoding device 

according to the invention. In Fig. 1, portions corresponding to those of the proposed device shown in Fig. 12 
are given the same reference numerals. 

Reference numeral 1 designates an input image which is input by the raster scan shown in Fig. 13, 12 code 
data which is encoded by an image signal encoding unit 100, and 102 second code data which is transformed 

•♦0 by a code transforming unit 101. The image signal encoding unit 100 encodes the input image 1 according to 
the method previously set forth in the description of the related art in the present specification and outputs 
the code data 12, and the code transforming unit 101 locally decodes the code data 12 and then encodes again 
the decoded data according to an algorithm different from the image signal encoding unit 100 to thereby trans- 
form the code thereof, and after then outputs the code data 102. 

45 The image signal encoding unit 100 shown in Fig. 1 is the same as has been previously set forth in the 

description of the related art in the present specification and thus the description thereof is omitted here. 

Fig. 2 is a block diagram showing the structure of the code transforming unit 101 shown in Fig. 1. In Fig. 
2. reference numeral 5 designates average value information which is separated from the code data 12 by a 
separation unit 200, 10 an approximate block which is separated from the code data 12 by the separation unit 

50 200, 8 mode information which is separated from the code data 12 by the separation unit 200, and 205 a locally 
decoded pixel block which is decoded by a locally decoding unit 204. The separation unit 200 separates the 
average value information 5, approximate block 10 and mode information 8 from the code data 12, the locally 
decoding unit 204 decodes the locally decoded pixel block 205 from the average value information 5, approx- 
imate block 10 and mode information 8. and a second encoding unit 206 encodes again the locally decoded 

55 pixel block 205 in accordance with the mode information 8 and then outputs second code data 102. 

Fig. 3 is a block diagram showing the structure of the locally decoding unit 204 shown in Fig. 2. 
In Fig. 3, reference numeral 211 designates inverse quantization control information which is output from 
a mode interpreter unit 210 so as to control the inverse quantization characteristic of an inversely quantizing 
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unit 213, 21 2 interpolation control information which is output from the mode interpreter unit 210 so as to control 
the interpolation characteristic of an interpolating unit 215, 214 a sub-sample decoded block which Is output 
from the inversely quantizing unit 213, and 216 an average value information separation decoded block which 
is output from the interpolater unit 215. The mode interpreter unit 210 interprets the mode information 8 and 

5 outputs the inverse quantization control information 211 and interpolation control information 212. the inversely 
quantizing unit 21 3 performs an inversely quantizing operation on the approximate block 1 0 in accordance with 
the inverse quantization control information 211, the interpolating unit 215 performs an interpolation processing 
on the sub-sample decoded block 214 in accordance with the interpolation control information 212, and an 
average value information adding unit 217 adds the average value information 5 to the averag'e- value sepa- 

10 ration decoded block 216 to thereby output the locally decoded block 205. 

Fig. 4 is a block diagram showing the structure of the second encoding unit 206 which is employed in Fig. 

2. 

In Fig. 4. reference numeral 221 designates a transformed coefficient block which is output from an or- 
thogonal transformation unit 220, 223 mask information which is output from a mask information generating 

15 unit 222 and is used to judge the significance/insignificance of the transformed coefficient block 221 , 225 a 
selected coefficient block which is judged for the significance/insignificance thereof by a coefficient selecting 
unit 224. and 227 a quantized coefficient block which is quantized by a quantizing unit 226. The orthogonal 
transformation unit 220 orthogonally transforms the locally decoded pixel block 205 and outputs the trans- 
formed coefficient block 221. the mask information generating unit 222 outputs the mask information 223 

20 showing the significance/insignificance of the coefficient for each coefficient position in correspondence to 
the mode information 8, the coefficient selecting unit 224 outputs the select coefficient block 225 with the 
coefficient at the coefficient position judged insignificant by the mask information 223 expressed at 0. the 
quantizing unit 226 divides the coefficients in the select coefficient block 225 by quantizing steps which are 
previously determined every coefficient position and outputs the quantized coefficient block 227. and an en- 

25 tropy encoding unit 228 entropy encodes the quantized coefficient block 227 to thereby obtain the second code 
data 102. 

Next, description will be given of the first embodiment of the invention with reference to Figs. 1 , 2, 3 and 
4. However, the operation of the image signal encoding unit 100 has been previously set forth in the description 
of the related art in the present specification and thus the description thereof is omitted here. 

30 As shown in Fig. 18, the code data 12, which is output from the image signal encoding unit 100, consists 

of the mode information 8, average value information 5, and approximate block 10. Further, the mode infor- 
mation 8 is composed of the waveform information 18 and gain information 27. In the separation unit 200 in 
Fig. 2, the mode information 8, average value information 5 and approximate block 10 are separated. 

In the mode interpreter unit 210 shown in Fig. 3, the waveform information 18 of the mode information 8, 

35 which is a resolution approximate parameter, is separated from the gain information 27 of the mode information 
8. which is a gradation approximate parameter. From the waveform information 18, a direction of interpolation 
can be determined uniquely in correspondence to a sub-sample pattern in encoding. There is obtained the 
interpolation control information 212 which controls the interpolation operation of the interpolating unit 215 in 
correspondence to the interpolation direction. Also, from the gain information 27 of the mode information 8, it 

40 is possible to determine the characteristic and the number of levels of the inverse quantization respectively 
corresponding to the number of levels of quantization uniquely. There is obtained the inverse quantization con- 
trol information 211 to control the inverse quantization operation of the inversely quantizing unit 213 in corre- 
spondence to the characteristic and level number of the inverse quantization. 

In the inversely quantizing unit 213 shown in Fig. 3, inverse quantization is performed on the approximate 

45 block 10 by use of the inverse quantization characteristic and level number which are notified by the inverse 
quantization control information 211, and the sub-sample decoded block 214 is output. The characteristic and 
level number of the inverse quantization are respectively prepared according to the adaptive quantizer 34 
shown in Fig. 16. 

In the interpolating unit 215 shown in Fig. 3, the sub-sample decoded block 214 is interpolated by means 
50 of an interpolation operation to be notified by the interpolation control information 212, and the average value 
separation decoded block 216 is output. Combinations of interpolation operations are prepared according to 
the sub-sample patterns of the adaptive sub-sampler 32 shown in Fig. 16. 

In the average value adding unit 217 shown in Fig. 3, the average value information 5 is added to the re- 
spective pixel values of the average value separation decoding block 21 6 to thereby output the locally decoded 
55 pixel block 205 of mxn pixels. Here, the locally decoded pixel block 205 is output in a block scan sequence 
shown in Fig. 5. 

Next, description will be given below of the second encoding unit 206 by use of Fig. 4. 

In the orthogonal transformation unit 220, two-dimensional orthogonal transformation is performed on the 
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locally decoded pixel block 205 of mxn pixels and the transformed coefficient block 221 is output. In Fig. 6, 
there are shown examples of pixel blocks and transformed coefficient blocks to be input. The numeral values 
of the pixel blocks respectively show the pixel values and the numeral values of the transformed coefficient 
blocks respectively show the values of the transformed coefficients. 

5 The mask information generating unit 222, in accordance with the waveform information 18. which is the 

resolution approximate parameter and the gain information 27. which is the gradation approximate parameter, 
respectively included in the mode information 8. outputs the mask information 223 which shows the signifi- 
cance/insignificance of the respective coefficients of the transformed coefficient block 221 . This is a process- 
ing which estimates the signal distribution of a frequency area from the spatial characteristic of fhe image>sig- 

10 nal. that is. the results of analysis of the resolution and gradation thereof. 

For example, as shown in Fig. 6(A). in a pixel block in which the gradation thereof varies in a horizontal 
direction, the coefficient electric power after the orthogonal transformation centers on the first row as shown 
in Fig. 6(a). Also, as shown in Fig. 6(B), in a pixel block in which the gradation thereof varies in a vertical di- 
rection, the coefficient power after the orthogonal transformation centers on the first column as shown in Fig. 

15 6(b). Such uniform gradation variations as in Fig. 6(A) and (B) are rare and the power is dispersed in other 
coefficients as well. However, when the horizontal- or vertical-direction gradation variations prevail, the coef- 
ficients in the neighborhood of the first row or in the neighborhood of the first column are respectively important 
in reproduction of the image quality. On the other hand, as shown in Fig. 6(C). when the edges exist in an 
oblique direction, the power is distributed in a wide range centering around the diagonal components of the 

20 transformed coefficient block as shown in Fig. 6(c). In this case, in reproduction of the image quality, the num- 
ber of the coefficients to be selected is increased when compared with those of the cases shown in Figs. 6(a) 
and 6(b). 

From the direction of the sub-sample, that is, from the directional property of interrelation between pixels 
in the pixel blocks, it is possible to know whether the electric power distribution in the transformed coefficient 
25 block 221 is in the row direction, in the column direction, or in the diagonal direction. Also, due to the fact that 
the sub-sample rate shows the intensity of interrelation, it can be understood that, as the sub-sample rate gets 
higher, the electric power centers more collectively on specific coefficients in a low frequency area. From the 
quantization characteristic and level, the existence and size of the edge in the pixel block can be estimated. 

As described above, in the mask information generating unit 222, from the sub-sample direction, sub-sam- 
30 pie rate, quantization characteristic, and quantization level respectively obtained from the mode information 
8, the mask information 223 showing the position of the coefficients necessary for reproduction of the image 
quality is decided and is output to the coefficient selecting unit 224. 

Description will be given below of the operation of the coefficient selecting unit 224 by use of Fig. 7. The 
mask information 223 shows whether each of the coefficients in the transformed coefficient block 221 is sig- 
35 nificant or insignificant, by means of "1" and "0" binary information for each coefficient position. As shown in 
Fig. 7, out of the coefficient block 221 , only the coefficients existing at the positions corresponding to "1" in 
the mask information 223 are selected to thereby form the selected coefficient block 225. 

The quantizing unit 226 quantizes the respective coefficients of the selected coefficient block 225 and 
outputs the quantized coefficient block 227. The quantizing processing can be shown by the following equation: 

40 



1f(u, v) - (£)(L', v) /2) /CJ(iJ, v) (F(u,v) <0) 

45 

where F(u,v) expresses the selected coefficient block 225, Q(u,v) expresses quantizing steps, C(u,v) express- 
es the quantized coefficient block 227, and u.v expresses a position in the block, respectively. 

The entropy encodingunit 228 encodes the quantized coefficient block 227 by means of entropy encoding 

50 such as Huffman encoding, arithmetic encoding or the like, and outputs the second code data 102. In Fig. 8, 
there is shown the quantized coefficient block 227. The coefficients in the block 227 are scanned in a zigzag 
within the matrix as shown by solid arrows in Fig. 8 to thereby provide a one-dimensional coefficient line, and 
an encoding processing is executed in this sequence. 

In the entropy encoding, at first, there are formed encoding symbols which are targets for code allocation. 

55 The symbols are formed individually by components according to the difference between the statistic proper- 
ties of the direct current component and alternating current component. In the code allocation as well, the en- 
coding is executed individually. For example, when performing the Huffman encoding, there are prepared in- 
dividual code tables such as a table for direct current components, a table for alternating current components 
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and the like. 

Description will be given below of a method of forming symbols for the direct current components. In the 
case of the direct current components, at first, there is obtained a difference between the direct current com- 
ponents of a previous block and a current block, and the resultant values are classified into groups according 

5 to Fig. 9. The numbers of the thus classified groups provide the encoding symbols. Also, the correspondence 
of the thus found difference values to the number of the groups is represented by a bit string (an additional 
bit) having the same number as the group number. 

Next, description will be given below of a method of forming symbols for the alternating current compo- 
nents. As shown in Fig. 8. the alternating current components except for the direct current components' are 

10 scanned in a zigzag within the block. In this case, the encoding symbols are formed by a combination of a 
length (zero run) in which invalid coefficients (zero coefficients) continue until a valid coefficient (of which value 
is not 0) appears, and the group number obtained when the valid coefficients are classified into groups (how- 
ever, the group 0 is not to be used) similarly to the direct current components. 

When the maximum length of the zero run is limited to 15 and the group number is also limited to 15. then 

15 there can be formed a table of 256 encoding symbols as shown in Fig. 10. Similarly to the case of the direct • 
current components, the actual correspondence of the coefficient values to the group number is indicate by 
the additional bit. 

When the Huffman encoding is used, by allocating codes according to the appearance probability of the 
symbols formed in the above-mentioned procedures, the second code data 102 is formed. 
20 Fig. 11 is a block diagram showing the structure of a second embodiment of the image signal encoding 

device according to the invention. In Fig. 11. a storage unit 103 is situated between the image signal encoding 
unit 100 and code transforming unit 101 to store the code data 12 temporarily. 

Due to provision of the storage unit 103, it is possible to transform only the part of the input image 1 into 
the second code data. Also, it is possible to perform an editing operation on the code data 12 on the storage 
25 unit 103. 

As has been described heretofore, according to the present invention, the following effects can be provid- 
ed. 

(1 ) Due to the fact that the code data can be re-encoded into the second code data of a higher compres- 
sibility by the code transforming unit, the data can be transmitted and stored efficiently. 
30 (2) Due to provision of the mask information generating unit, there can be generated the mask information 

which is used to discriminate between the significant and insignificant coefficients of the transformed coef- 
ficient block in accordance with the mode information, and the coefficient selection is made in the coef- 
ficient selecting unit. This can improve the image quality deterioration of the edge portion. 

(3) By employing as the second encoding unit a well-known encoding technology based on transformation 
35 encoding, there can be secured the changeability of data with other image. processing devices. 

(4) Since a second encoding operation in a frequency space is performed by a first encoding operation in 
a pixel space, the first encoded information does not need to be restored but, while the first encoded in- 
formation is left as it is, an encoding operation can be performed in the next stage, so that a two-stage 
high compression can be executed in a simple structure. Here, it is noted that the common processing 

40 unit is employed in the first and second encoding operations and the first encoded signal is supplied directly 

to the second encoded signal. Namely, the common processing block is determined under the condition 
that it represents the distribution/dispersion condition of the characteristic distribution of image and it is 
suited for the real-time processing. Accordingly, for example, even if the first encoded signal is once stored 
in a page memory, the real-time processing can be performed. Thus, the two-stage high compression can 

45 be executed entirely in a real-time manner and at a high speed. As has been described above, in the in- 

vention, the first and second encoding algorithms which are different from each other are related so as to 
be compatible with accuracy and compressing ratio. This could not be accomplished in the conventional 
techniques. 

(5) Due to the fact that the first encoding is executed by mode discrimination in the pixel space and the 
50 second encoding is executed by making use of the mode discrimination results for processing coefficients 

in a processing space, it is possible to transform the second encoding in the frequency space with accuracy 
in a simple mode discriminating structure, without adding any means for the second encoding operation. 
Here, it is noted that the common processing block unit is employed in the first and second encoding op- 
erations. Namely, since the two-stage compression is executed for the common processing block, high 
55 compression can be accomplished without degrading the image quality. 

(6) Since the second encoded results do not contain the mode discrimination information, a special re- 
ceiving device is not required. 

(7) For example, when image information is to be transmitted in the JPEG international standard system 
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from a device which is provided at a transmitting side and has a compression function, it is possible to re- 
duce Image deterioration due to the second encoding in a frequency space of a device at the international 
standard system side which does not have a mode discrimination function. Also, as the receiving device, 
any device can be used, provided that it is in conformity with the JPEG international standard system. 
5 Thanks to the above-mentioned effects, by combining the present invention with the proposed encoding 

method which aims at reducing a page memory, there can be realized an efficient image data encoding meth- 
od. 

Also, by providing the memory unit between the image signal encoding unit and the code, transforming 
unit, the partial code transformation of an image as well as the code transformation after the editmg processing 
10 are possible, which saves a processing on the unnecessary image data. 



Claims 

15 1. An Image signal encoding device comprising: 

dividing means for dividing an image signal into a plurality of rectangular pixel blocks: 
average value separator means for calculating an average value of pixels in each of the pixel blocks 
and subtracting the average value from each of the values of the pixels of the pixel block: 

mode discriminator means for analyzing characteristics of resolution and gradation variations In an 
20 average value separation pixel block obtained by said average value separator means and outputting 

mode information which Is defined In correspondence to the characteristics of the pixel distribution In the 
average value separation pixel block: 

first encoding means for encoding the average value separation pixel block by means of an en- 
coding operation which is defined by the mode information; 
25 multiplexer means for multiplexing the encoded results output from said first encoding means, the 

average value output from said average value separation means, and the mode information and outputting 
first code data: 

separation means for separating the mode Information from the first code data: 
locally decoding means for locally decoding the first code data by means of a decoding operation 
30 corresponding to the mode Information and outputting a locally decoded pixel block; and 

second encoding means for encoding the locally decoded pixel block by means of an encoding op- 
eration corresponding to the mode information and outputting second code data. 

2. The image signal encoding device as setforth In claim 1. wherein said second encoding means comprises; 

35 orthogonal transformation means for orthogonally transforming the locally decoded pixel blocks In a two- 

dimensional manner and outputting a transformed coefficient block: mask information generating means. 
In accordance with the mode information, for outputting as mask Information binary patterns which are 
used to judge the significance/Insignificance of the respective coefficients in the transformed coefficient 
block: coefficient selecting means forjudging the significance/insignificance of the respective coefficients 

40 In the transformed coefficient block by use of the mask information and outputting a selected coefficient 

. block In which the transformed coefficients of the transformed coefficient block that are judged insignif- 
icant are expressed as 0: quantizing means for quantizing the respective coefficients of the selected coef- 
ficient block and outputting a quantized coefficient block; and entropy encoding means for entropy en- 
coding the quantized coefficient block and outputting the second code data. 



45 



50 



55 



3. The Image signal encoding device as set forth in claim 1, further comprising storage means for storing 
the first code data output from said multiplexer means between said multiplexer means and said separa- 
tion means. 

4. The image signal encoding device as set forth in claim 2. further comprising storage means for storing 
the first code data output from said multiplexer means between said multiplexer means and said separa- 
tion means. 

5. An image signal encoding device comprising: 

mode discriminator means for discriminating mode information of an image signal for each of pixel 
blocks in a pixel space; 

first encoding means for encoding image data of the pixel blocks in accordance with modes of said 
mode discriminator means; 
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transforming means for expressing the first encoded information by a basic waveform in a frequen- 
cy space of the image signal and a coefficient for the basic waveform: 

coefficient switching means for switching the coefficient in the frequency space in accordance with 
the modes of said mode discriminator in the pixel space; and 

second encoding means for encoding, in the frequency space, the encoded results with respect 
to the image data of the pixel block that are output from said first encoding means in accordance with a 
coefficient set by said coefficient switching means. 

The image signal encoding device as set forth in claim 5. further comprising storage means for storing 
the encoded results from said first encoding means. 

The image signal encoding device as set forth in claim 5, wherein the encoded results from said first en- 
coding means are in the form of data which include the mode information, and the average value and 
approximate block information of the pixel block. 

A method of encoding an image signal, comprising the steps of: 

discriminating mode information for each of pixel blocks of the image signal in a pixel space: 

first encoding image data of the pixel block in the pixel space in accordance with the modes ob- 
tained in said discriminating step: 

expressing the first encoded information by a basic waveform in a frequency space of the image 
signal and a coefficient for the basic waveform: 

switching the coefficient in the frequency space in accordance with the modes obtained in said 
discriminating step in the pixel space: and 

second encoding the encoding results in said first encoding step with respect to the image data of 
the pixel block in accordance with a coefficient set in said switching step. 
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(A) PIXEL BLOCK 1 (HORIZONTAL GRADATION] (a) COEFFICIENT DISTRIBUTION IN OCT 
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(3) PIXEL BLOCK 2 (VERTICAL GRADATION) (b) COEFFICIENT DISTRIBUTION IN OCT 
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(C) PIXEL BLOCK 3 (OBLIQUE EDGE) (c) COEFFICIENT DISTRIBUTION IN DGT 

TRANSFORHATION OF PIXEL BLOCK 3 



16 



EP 0 584 741 A2 



FIG. 7 



221 

OOOOOxxO 
OOOOxOxO 
OOOOx X X X 
OOOOx X X X 
OOOxOx X X 
OO X X X X X X 
xOxxxxxx 
Oxxxxxxx 



TRANSFORMATION COEFFICIENT 



rO: 

I X : 



NON-ZERO COEFFICIENT 
ZERO COEFFICIENT 



1 1 10 0 0 0 

110 0 0 0 0 

1 10 0 0 0 0 

110 0 0 0 0 

1 0 0 0 0 0 0 

1 0 0 0 0 0 0 

1 0 0 0 0 0 0 

1 0 0 0 0 0 0 



223 



COEFFICIENT 
SELECTING UNIT 



T 



^ 225 

OOOOxxxx 
OOO X X x x X 
OOOx X XXX 
OOOx X x X X 
OOx X X X X X 
OO x X X X X X 
xOx X X X X X 
Oxxxxxxx 



224 SELECTED TRANSFORMATION 
COEFFICIENT 



FIQ. 8 



D. C. COMPONENTS 



HORIZONTAL DIRECTION 
FREQUENCY COMPONENTS 



LOW 



OO 

z z 

O LU 

— z 

I— o 
cj a_ 
<_ij z 
az o 

i— . CJ 
. C3 

>- 
o 
<c z 

O UJ 

I— a 

CC LU 

UJ c: 



HIGH 



LOW 



HIGH 





















































































1 































227 



17 



EP 0 584 741 A2 



FIG. 9 



GROUP NO. 
(ENCODING SYMBOL) 

0 
1 

2 
3 
4 
5 
8 
7 
8 
9 

10 
1 1 
12 
13 
14 
15 



DIFFERENCES IN 
0. C- COMPONENIS 



-7 
-15 
-31 
-63 
-127 
-255 
-511 
-1023 
-2047- 
-4095- 
-8091 • 
-16383- 
-32767- • 



0 

-1. 1 
3. -2. 2. 3 
•. -4, 4. • 
• , "8. 8, • 
. -16. 16, 
. -32. 32. 
. -64. 64. 
-128. 128 
-256. 256 
-512. 512 
-1024. 1024 
-2048. 2048 
-4096. 4096 
-8092. 8092 
16384. 16384 



7 

15 
31 
63 
127 



255 

511 

1023 

•2047 

•4095 

•8091 . 

•16383 

••32767 



NUMBER OF 
ADDITIONAL 

0 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 



BITS 



FIG. 70 



RUN 



SIZE 



16 



32 



17 



19 



ENCODING SYMBOL DEPENDING ON 
COMBINATION OF RUN AND SIZE 



15 



240 



255 



SIZE : GROUP NUMBER OF LEVEL VALUES OF VALID COEFFICIENTS 
RUN : RUNS OF INVALID COEFFICIENTS FROM 0 TO 15 

HOWEVER. WHEN SIZE = 0. 

RUN =0 : END OF' BLOCK 

=15 : 16 INVALID COEFFICIENT RUNS 

(FOR 16 OR MORE RUNS, THIS WILL BE REPEATED) 
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